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United  States  Air  Force 

Battlestaff  Training  School  and  BLUE  FLAG  Exercises: 
Capabilities  and  Challenges 


Joint  Force  Air  Component  Commander’s 
(JFACC)  Mission 

The  United  States  Air  Force  Battlestaff  Training 
School  (USAFBTS)  provides  command  and 
control  (C2)  training  for  the  Joint  Force  Air 
Component  Commanders  (JFACC)  and  their 
staffs  who  man  the  Joint  Air  Operations  Centers 
(JAOC).  A  JFACC  is  a  senior  officer  from  one 
service  branch,  such  as  Air  Force  or  Navy,  who 
is  responsible  for  ail  air  power  employment  in  a 
specific  theater  of  operations.  The  JFACC  is 
appointed  by  the  theater  Commander  in  Chief 
(CINC)  or  Joint  Task  Force  (JTF)  Commander. 
The  JAOC  is  composed  of  the  JFACC,  his  joint 
staff  (including  Air  Force,  Army,  Navy,  Marines, 
and  special  forces),  and  coalition  representatives 
from  the  specific  theaters  of  operation.  The 
JFACC  is  responsible  for  all  allied  air  operations 
in  a  theater  of  war,  and  produces  the  Air  Tasking 
Order  (ATO)  for  each  day  of  the  war. 

Some  of  the  constraints  in  training  the  JFACC 
and  his  staff  include: 

1)  Overwhelming  real-world  commitments. 

2)  Nonstandard  command,  control,  commun¬ 
ications,  computers,  and  intelligence  (C4I) 
equipment  between  theaters  of  operations  and 
service  components. 

3)  Frequent  personnel  turnover. 

4)  Peace-time  manning  levels  require  augment¬ 
ation  for  training  and  deployments. 

In  order  to  minimize  the  impact  of  these 
constraints,  the  Numbered  Air  Forces  (NAF) 
participate  in  highly  intense,  robust  training 
exercises.  RED  FLAG  exercises  provide  tactical- 
level  training  via  live  flight  missions  over  the 
Nellis  AFB,  NV  ranges;  GREEN  FLAG  exercises 
build  on  a  RED  FLAG  by  incorporating  the 


electronic  warfare  aspect  of  combat;  BLUE 
FLAG  exercises  train  C4I  procedures  at  the 
operational/theater  level  of  warfare. 

Battlestaff  Training  School’s  Mission 

The  USAF  Battlestaff  Training  School,  located  at 
Hurlburt  Field,  FL,  is  part  of  the  505th  Command 
and  Control  Evaluation  Group,  a  subordinate  unit 
of  the  53d  Wing,  Eglin  Air  Force  Base,  FL  and 
the  Air  Warfare  Center,  Nellis  AFB,  NV.  The  Air 
Warfare  Center  is  a  component  of  Air  Combat 
Command,  located  at  Langley  AFB,  VA.  Air 
Combat  Command  is  responsible  for  organizing, 
training,  and  equipping  the  combat  air  forces 
within  the  United  States.  The  Battlestaff  Training 
School  is  tasked  by  Air  Combat  Command  to 
conduct  a  series  of  BLUE  FLAG  exercises  each 
year  to.  train  combat  leaders  and  supporting 
battlestaff  personnel  in  command,  control, 
communications,  computer,  and  intelligence 
procedures  for  specific  theaters  of  operation. 

The  USAFBTS  mission  is  to  train  the  JFACC 
team  through  realistic,  computer-assisted 
exercises  focused  on  air  power  employment. 
The  school’s  vision  is  to  feature  world-class 
command  and  control  experts,  training 
battlestaffs  to  achieve  information  dominance 
while  optimizing  air  and  space  power 
employment.  USAFBTS  hosts  BLUE  FLAG 
exercises  to  train  America’s  Numbered  Air  Force 
staffs  as  the  core  for  the  JFACC  staff.  Although 
the  primary  training  audience  is  the  NAF  staff, 
other  services  and  coalition  forces  receive 
valuable  training  during  BLUE  FLAG  exercises. 
Representatives  from  the  Army,  Navy,  Marines, 
Special  Forces,  and  coalition  forces  all 
participate  as  valuable  components  of  the  Air 
Operations  Center  (AOC).  The  Battlestaff 
Training  School  has  the  capability  to  develop  a 
scenario  for  any  situation  or  location  for  its  BLUE 
FLAG  exercises. 
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Key  to  BLUE  FLAG  exercise  success  is  the  BTS 
training  philosophy.  The  goal  is  to  make  the 
wargame  as  realistic  as  possible,  such  that 
participants  cannot  distinguish  it  from  combat. 
The  school  accomplishes  this  by  replicating  the 
actual  C2  structure,  using  real-world  equipment, 
and  using  knowledgeable  exercise  controllers  to 
replicate  the  enemy  and  all  other  agencies  not 
physically  represented  in  the  game.  A  key 
component  of  the  BLUE  FLAG  training  is  to 
facilitate  interaction  among  the  participants. 

BLUE  FLAG  Exercises 

BLUE  FLAG  exercises  are  Air  Combat 
Command’s  foremost,  large-scale,  force-on- 
force,  computer-assisted,  airpower  exercises. 
They  provide  joint  service  C2  battlestaff  training 
by  emphasizing  real-world  plans,  procedures, 
and  C4I  equipment.  The  NAF  commander  and 
his  staff  develop  training  objectives  that  drive 
each  exercise.  Several  planning  conferences  are 
held  during  a  six-month  planning  cycle.  This 
allows  the  USAFBTS  staff  and  the  NAF  staff  to 
work  closely  in  designing  the  exercise  to  meet 
the  training  objectives.  It  is  the  sole 
responsibility  of  the  JFACC  and  his  staff  to 
determine  if  training  objectives  have  been  met. 
USAFBTS  provides  a  copy  of  the  lessons 
learned  and  a  final  report  to  the  NAF.  These 
reports  are  a  compilation  of  inputs  from  senior 
players,  controllers,  and  USAFBTS  staff.  More 
will  be  said  about  potential  ways  to  improve  the 
feedback  process  below. 

Each  BLUE  FLAG  exercise  begins  with  two  to 
three  days  of  computer,  academic,  and  seminar 
training,  focusing  on  the  skills  needed  to  man  a 
fully  functioning  Air  Operations  Center.  This  is 
followed  by  a  four-  to  five-day,  computer- 
assisted  exercise  using  actual  friendly  and 
enemy  orders  of  battle,  contingency  and  war 
plans,  and  theater  procedures  in  a  simulated 
military  operation.  Participants  are  given 
maximum  flexibility  in  managing  the  employment 
of  friendly  forces  thus  allowing  player  actions  to 
influence  the  battle  outcome. 

BLUE  FLAG  exercises  attempt  to  duplicate 
theater  conditions  and  procedures  with 
maximum  fidelity.  USAFBTS  planners  research 
friendly  and  enemy  force  structures, 
communication  capabilities,  logistics  support, 


command  and  control  procedures,  and  current 
plans  and  directives.  USAFBTS  relies  partially 
on  theater  advisors  for  validation  of  exercise 
scenarios  and  proper  location  of  participating 
headquarters  and  wartime  command  and  control 
structures.  Theater  advisors  may  also  assist  with 
some  player  training  and  act  in  non-player, 
higher  headquarters  control  functions. 

USAFBTS  utilizes  four  primary  computer  models 
during  BLUE  FLAG  exercises  to  interface  with 
operational  C4I  systems.  Each  of  these  models 
replicates  reality  in  a  distinctive  way  and  uses 
automated  interfaces  and  information 
management. 

The  first  of  these  models  is  the  Air  Warfare 
Simulation  (AWSIM).  AWSIM  allows  the  school 
staff  to  model  a  wide  range  of  airborne  platforms 
and  ground-based  defensive  systems.  AWSIM 
also  flies  airborne  platforms  and  fires  surface-to- 
air  missiles  in  real  time. 

The  second  model,  the  Theater  Exercise 
Intelligence  Simulator  (TEXIS),  replicates  only 
ground-based  objects  such  as  radars,  buildings, 
airfields,  surface-to-air  missile  (SAM)  sites  and 
mobile  targets.  Objects  are  loaded  into  the 
TEXIS  database  so  that  actual  mission  results 
and  reports  can  be  generated  from  an  AWSIM- 
flown  attack.  TEXIS  replicates  15  recon¬ 
naissance  collection  platforms  to  observe  these 
objects. 

A  third  model,  the  National  Wargaming  System 
(NWARS),  simulates  national  reconnaissance 
assets.  It  interfaces  with  TEXIS  to  generate 
reports  and  information  for  player  analysis.  More 
importantly,  it  teaches  players  the  national 
reconnaissance  request  process. 

Our  fourth  model  is  the  Joint  Electronic 
Combat/Electronic  Warfare  Simulation 
(JECEWSI),  an  electronic  combat  model. 
Through  JECEWSI’s  interface  with  AWSIM,  the 
BTS  staff  accurately  models  the  force  multiplying 
nature  of  electronic  combat  aircraft  like  the  EF- 
111  and  EC-130. 

Overall,  these  systems  help  BLUE  FLAG 
controllers  fly  the  player-developed  ATO  against 
a  thinking  enemy.  By  wisely  using  these  tools, 
the  BTS  staff  maintains  a  high  degree  of  realism 
throughout  the  wargaming  phase  of  training. 
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More  than  40,000  personnel  from  all  branches  of 
the  military  have  participated  in  BLUE  FLAG 
training.  The  first  BLUE  FLAG  exercise  was 
conducted  in  December  1976  at  Shaw  Air  Force 
Base,  SC.  The  original  exercises  involved  live 
flying  operations.  However,  in  1979,  the 
simulation  exercise  format  was  adopted  to 
emphasize  the  primary  training  objective  of  battle 
commanders  exercising  an  integrated  war 
campaign. 

In  1984,  BLUE  FLAG  began  its  evolution  from 
the  grease  pencil  into  the  computer  age.  The 
first  step  was  to  employ  the  Tactical  Simulation 
(TACSIM)  to  portray  ground  and  intelligence 
orders  of  battle.  In  1985,  the  Air  Defense 
Simulation  (ADSIM)  was  incorporated  to 
replicate  friendly  air  defense  forces  against 
enemy  air  forces.  Through  the  next  five  years, 
continuous  improvements  were  made  in  both 
simulations  to  meet  growing  needs  for  a  truer 
depiction  of  battle. 

The  year  1990  produced  quantum  advances  in 
capabilities  to  portray  a  theater  war.  TACSIM 
was  improved  and  modified  to  such  an  extent  it 
no  longer  resembled  the  original  program  and 
was  renamed  the  Theater  Exercise  Intelligence 
Simulation,  orTEXIS,  to  represent  ground  orders 
of  battle  and  to  produce  mission  results  and 
reconnaissance  reports.  ADSIM  no  longer  met 
the  need  for  the  air  war  and  was  replaced  with 
the  Air  Warfare  Simulation,  or  AWSIM,  to  better 
portray  the  air  war.  This  new  software  allows 
both  offensive  and  defensive  activities  to  be 
executed  simultaneously. 

Also  in  1990,  the  USAFBTS  accomplished  a  first 
for  a  unit  located  in  the  United  States  by 
conducting  a  distributed  wargame,  Warrior  Flag 
90.  This  exercise,  using  distributed  wargaming 
system  (DWS)  equipment,  demonstrated  the 
technology  to  distribute  training  and  secure 
video  teleconferencing  simultaneously  to  multiple 
remote  locations  on  separate  continents.  In 
January  1992,  BLUE  FLAG  acquired  a  five-site 
remote  capability  to  augment  its  DWS  exercise 
function. 

The  Contingency  Theater  Automated  Planning 
System  (CTAPS)  was  fully  integrated  into  BLUE 
FLAG  exercises  in  1993,  further  enhancing  the 
USAFBTS  reputation  for  accurately  replicating 
real-world  command  and  control  systems  in  Air 


Operations  Center  and  Joint  Force  Air 
Component  Commander  training.  USAFBTS  has 
hosted  CTAPS  testing  of  improved  versions 
being  readied  for  operational  fielding. 

In  January  1995,  Gen  Fogleman,  USAF  Chief  of 
Staff,  sent  a  new  vector  challenge  for  Air  Force 
simulation  to  all  Major  Commands-specifically  to 
accelerate  insertion  of  advanced  simulation 
technology  into  our  training  and  testing. 
WARFIGHTER  95  was  the  first  major  step  in  that 
direction.  The  purpose  was  to  validate  advanced 
distributed  simulation  (ADS)  technology  for 
application  into  future  BLUE  FLAG  exercises 
and  JFACC  team  training.  This  exercise 
provided  several  new  technologies  that  have 
been  incorporated  into  our  current  day  BLUE 
FLAG  exercise  suite.  Using  Distributed 
Interactive  Simulation  (DIS)  to  stimulate  a  variety 
of  models  and  simulations,  the  real-world  air 
picture  is  now  provided  via  Tactical  Digital 
Information  Link  (TADIL)  and  Tactical 
Information  Broadcast  Service  (TIBS)  into  the 
AOC.  The  Joint  Synthetic  Battlespace  is  the 
goal  of  the  future  and  will  eventually  tie  together 
many  types  of  simulation  where  warfighters  and 
analysts  will  be  able  to  plug  into  a  common 
battlespace  from  desks,  simulators,  aircraft,  and 
crew  stations,  linking  services  and  civilian 
counterparts  to  train  and  conduct  warfighting. 

USAFBTS  has  received  direction  from  higher 
headquarters  to  improve  training  in  the  following 
areas: 

1)  Increase  realism  in  painting  the  battle  situation 
keep  “fog  of  war”  and  “blizzard  of  information” 
transition  from  sensor  scripting  to  displaying 

2)  Include  subordinate  C2  elements  in  Airborne 
Warning  and  Control  System  (AWACS)  and 
Airborne  Command  and  Control  Center 
(ABCCC)  aircrews  and  Air  Command  Element 
(ACE)  teams  Control  and  Reporting 
Centers/Elements  (CRC/CRE) 

3)  Include  selected  events  from  the  tactical  level 

4)  Merge  constructive,  virtual,  and  live 
operations 

Within  the  past  year  the  Battlestaff  Training 
School  has  been  able  to  triple  the  number  of 
intelligence  messages  sent  during  an  exercise, 
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but  this  number  still  falls  short  of  the  message 
traffic  in  actual  combat.  Even  this  level  of  3,000 
messages  per  day  overwhelms  operators  trying 
to  find  key  information  amid  the  myriad  data. 
Making  the  transition  from  scripting  sensor 
reports  to  displaying  sensors  in  action  is  a 
significant  challenge.  The  key  is  having  a 
theater-level  ground  database  that  can  be  seen 
by  sensor  models,  and  meet  the  requirements 
for  doctrinally  sound  engagements,  damage 
reporting,  and  execution  speed.  The  school  is 
not  there  yet.  Promising  solutions  are  being 
worked,  including  the  JC2WC-developed  Joint 
Operations  Information  Simulation  (JOISIM)  and 
the  longer  range  Joint  Simulation  System 
(JSIMS). 

This  past  year  BTS  expanded  the  participation  to 
C2  units  immediately  subordinate  to  the  air 
operations  center  (i.e.,  AWACS,  CRE,  CRC). 


Modular  Control  Element  (MCE)  units  were 
manned  by  the  505  Test  Support  Squadron  to 
act  as  CRE  and  CRC  platforms.  The  AWACS 
simulator,  located  at  Tinker  AFB,  OK, 
participated  as  a  distributed  site  providing 
datalink  messages  to  the  combined  air  picture. 

WARRIOR  FLAG  97  was  the  next  step  in 
creating  the  Chief  of  Staff’s  vision  of  a  Joint 
Synthetic  Battlespace.  Live,  Virtual,  and 
Constructive  simulation  were  combined  to  create 
an  enhanced  BLUE  FLAG-type  exercise. 
WARRIOR  FLAG  was  our  chance  to  test  out 
new  technologies  without  interfering  with  a  BLUE 
FLAG  training  exercise.  Technology  improve¬ 
ments  that  prove  to  be  valuable  training  aids 
were  injected  into  future  BLUE  FLAG  exercises. 
Figure  1  shows  the  major  assets  involved  in 
producing  Warrior  Flag  ’97. 


Figure  1.  Assets  involved  in  Warrior  Flag  '97  exercise 


The  diagram  above  depicts  the  general  combat 
elements  that  made  up  WARRIOR  FLAG  97. 
Live  aircraft  equipped  with  Global  Positioning 
System  (GPS)  relay  pods  flew  over  the  Eglin 


AFB,  FL,  ranges  and  sent  their  positional  data  to 
computers  which  converted  their  position  to  DIS 
protocol  data  units  (PDUs).  These  were  then 
displayed  in  a  common  air  picture  with  the 
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constructive  and  virtual  models.  We  used  the 
same  architecture  as  we  use  in  a  BLUE  FLAG 
exercise.  This  architecture  was  enhanced  by  the 
live  injects  and  by  DIS-compliant  virtual 
simulators.  A  portion  of  the  ATO  was  flown  out 
by  virtual  cockpits  located  at  the  Air  Force 
Research  Laboratory’s  Aircrew  Training 
Research  Division  in  Mesa,  AZ,  and  by  virtual 
simulators  in  Albuquerque,  NM,  and  Washington 
DC.  All  of  these  inputs  were  displayed  in  a 
common  fused  air  picture  and  monitored  by  the 
JFACC  and  his  AOC  staff  located  at  Hurlburt 
Field,  FL. 


various  models  and  simulations  operating  on 
local  area  networks,  the  Aggregate  Level 
Simulation  Protocol  and  Distributed  Interactive 
Simulation  protocols  have  been  the  primary 
means  of  wide-area  network  communication  in 
recent  years.  Unfortunately,  both  of  these 
protocols  represent  “work  in  progress”  that  has 
not  been  fully  developed.  The  near-term  impact 
of  the  transition  to  High  Level  Architecture  on  the 
integration  of  live,  virtual,  and  constructive 
simulations  as  well  as  its  impact  on  the 
continuing  development  of  communication 
protocols  remains  to  be  seen. 


Feasibility  and  Value  of  Linking  BLUE  FLAG 
Exercises  with  Live  and  Virtual  Assets 

Technology  Issues 

As  we  move  toward  implementing  the  Joint 
Synthetic  Battlespace  envisioned  by  the  Air 
Force  Chief  of  Staff,  there  will  be  increasing 
horizontal  and  vertical  integration  of  the  different 
classes  of  models  and  simulations.  This  will 
provide  revolutionary  synthetic  environments, 
containing  various  mixes  of  computer  models, 
weapon  system  simulators,  and  actual 
warfighting  systems.  The  entities  within  these 
environments  will  be  geographically  dispersed 
and  will  communicate  through  high-speed 
communications  networks.  The  result  will  be  a 
hierarchy  of  models  and  simulations  to  support 
such  functional  areas  as  training,  test  and 
evaluation,  analysis,  and  decision  support. 

Technical  progress  is  readily  apparent 
There  have  been  numerous  instances  in  which 
the  various  combinations  of  live,  virtual,  and 
constructive  simulations  have  been  linked 
together  to  support  technical  demonstrations, 
concept  development,  operational  tests,  and 
training.  There  are,  however,  still  a  number  of 
challenges  associated  with  creating  the  Joint 
Synthetic  Battlespace.  These  challenges  involve 
the  development,  integration,  and  fidelity  of 
appropriate  models  and  simulations,  commun¬ 
ication  between  geographically  separate  sites, 
eand  xecution  of  geographically  distributed 
training  involving  different  training  audiences. 
These  technical  challenges  include: 

Communication  protocols.  Although  there  are 
a  number  of  unique  protocols  used  to  link 


Fidelity.  Constructive  simulations  generally 
present  the  individual  weapon  system  and  the 
warfighter’s  behavior  at  a  lower  level  of  fidelity 
than  either  live  or  virtual  simulations. 
Consequently,  entities  represented  by  live  and 
virtual  simulations  typically  display  behaviors 
that  are  more  variable,  complex,  and  realistic 
than  those  exhibited  by  constructive  entities.  For 
example,  live  or  virtual  entities  are  typically  more 
likely  to  have  their  performance  disrupted  by 
fatigue,  stress,  or  workload  than  constructive 
entities. 

Bandwidth.  The  use  of  live  and  virtual 
simulators  usually  means  there  is  a  requirement 
for  close  coupling  between  at  least  some  of  the 
entities.  Generally  speaking,  these  closely 
coupled  systems  are  high  fidelity,  human-in-the- 
loop  entities  that  demand  a  higher  degree  of 
interactivity  than  typical  constructive  entities.  To 
support  this  higher  degree  of  interactivity,  data  is 
typically  exchanged  more  often.  Thus,  for  a 
given  network  bandwidth,  a  few-on-few 
engagement  involving  live  or  virtual  platforms 
could  produce  the  same  network  loading  as  a 
many-on-many  battle  comprised  solely  of 
constructive  entities. 

Command  and  control.  There  are  a  number  of 
command  and  control  challenges  involved  in 
“seamlessly”  integrating  live,  virtual,  and 
constructive  entities.  For  example,  there  are 
real-world  constraints  on  the  actual  weapon 
system  that  must  be  managed  as  part  of  the 
simulation.  Live  weapon  systems  must  adhere 
to  the  range  safety  rules,  air  traffic  control,  and 
airspace  restrictions.  As  a  result,  the  rules  of 
engagement  between  live  entities  may  be 
significantly  different  from  those  for  either  virtual 
or  constructive.  Another  challenge  involves  the 
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fact  some  interactions  are  logically  impossible 
between  the  various  classes  of  entities.  For 
example,  although  a  live  entity  can  engage  either 
a  virtual  or  constructive  entity  beyond  visual 
range,  there  is  no  realistic  way  to  allow  virtual  or 
constructive  entities  to  become  part  of  a  close-in, 
visual  engagement. 

Aggregation.  Different  models  and  simulations 
operate  at  different  levels  of  granularity.  Live 
and  virtual  simulations  represent  individual 
warfighters  and  their  weapon  systems.  To 
adequately  support  these  individual  warfighters, 
detailed  platform  dynamics,  terrain  resolution, 
time  steps,  and  weapons  effects  are  essential. 
On  the  other  hand,  constructive  simulations 
often  represent  aggregated  units  without  detailed 
modeling  of  individual  platforms  or  terrain. 
Algorithms  representing  typical  behaviors, 
weapons  effectiveness,  and  attrition  are  used  to 
produce  aggregated  combat  results.  Because 
the  level  of  granularity  differs  between 
constructive  and  live/virtual  simulations,  it  is 
necessary  to  aggregate  the  live/virtual  forces 
into  appropriately  sized  units  and  to  incorporate 
the  results  of  live/virtual  simulations  into  the 
various  engagement  and  attrition  models  used 
by  the  higher  level  model. 

Sensor  data.  The  representation  and 
transmission  of  sensor  data  such  as  radar  cross 
section,  radar  modes,  and  infrared  signatures  is 
one  of  the  major  issues  in  Advanced  Distributed 
Simulation.  The  accurate  communication  of  such 
data  within  virtual  simulations  as  well  as  between 
the  various  classes  of  simulations  is  essential  for 
depicting  the  Joint  Synthetic  Battlespace 
accurately. 

Training  Value.  The  goal  of  training  is  to 
improve  the  performance  of  personnel.  The  Joint 
Synthetic  Battlespace  provides  an  enriched 
environment  for  mission-oriented  training.  The 
generic  structure  of  this  environment  is  shown  in 
Figure  2.  This  environment  assumes  mission 
plans,  doctrine,  and  tactics  are  combined  with  an 
appropriate  training  philosophy  to  create  new  or 
improved  training  opportunities.  The  integration 


of  live,  virtual,  and  constructive  simulations 
provides  an  opportunity  for  warfighters  to 
improve  their  skills  at  the  engagement  level  while 
commanders  and  staffs  are  improving  their 
decision  making  and  management  skills.  These 
performance  improvements  may  be  reflected  in 
improved  use  of  weapon  systems,  better  team 
work  and  coordination,  or  improved  decision 
making. 

Although  the  idea  of  integrated  training  within  the 
Joint  Synthetic  Battlespace  has  a  great  deal  of 
intuitive  appeal,  detailed  training  needs  and 
media  analyses,  training  syllabi,  and  training 
effectiveness  measures  have  not  been 
developed.  Instead,  the  general  approach 
seems  to  be  predicated  on  the  notion  that  if  we 
can  successfully  create  the  mission 
environment,  we  will  be  able  to  improve  training. 
While  this  may  be  true,  there  are  a  number  of 
inherent  training  risks  involved.  All  other  things 
being  equal,  we  know  the  more  time  individuals 
spend  practicing  a  task  and  the  better  the 
feedback  those  individuals  receive  about  their 
performance,  the  better  their  performance.  One 
of  the  potential  problems  in  combining  a  number 
of  different  echelons  within  the  same  simulation 
exercise  is  the  different  echelons  operate  along 
different  time  scales  and  have  different 
objectives.  As  a  result  of  these  differences, 
individual  warfighters  may  be  forced  to  “hurry  up 
and  wait”  until  their  opportunity  to  perform  at  the 
engagement  level  occurs.  The  net  result  of  this 
is  their  time  on  mission-critical  tasks  is  not 
optimized.  Another  problem  is  the  level  of  detail 
concerning  individual  performance  differs  across 
echelons.  Individual  warfighters  in  live  or  virtual 
simulators  typically  require  detailed  feedback 
and  review  of  individual  decisions  and  actions  at 
a  number  of  different  points  within  their  mission. 
The  JFACC  and  his  staff,  on  the  other  hand,  are 
rarely  concerned  about  individual  performance. 
Instead,  they  must  assess  the  impact  of 
aggregate  performance  on  mission 
accomplishment  and  future  mission  plans. 
Figure  2  depicts  the  integration  of  live,  virtual, 
and  constructive  assets  for  command  and 
control  training. 
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Figure  2  -  Integration  of  Live,  Virtual,  and  Constructive  Assets  for  Command  and  Control  Training 


Potential  Techniques  for  Improving 
BLUE  FLAG  Exercise  Feedback  to  the 
Joint  Force  Air  Component  Commander 
and  Staff 


Current  Feedback  Challenges 

Quality  and  Value  -  As  described  above, 
the  ratio  of  BTS  controllers/observers  compared 
to  BTS  participants  is  quite  low.  It  is  not  possible 
for  the  BTS  analysts  to  observe  and  catalog 
each  error  committed  by  individuals,  teams,  and 
the  entire  JFACC  staff  due  to  the  number  and 
rapidity  of  activities  occurring  minute  by  minute. 
In  addition,  these  activities  often  occur  in  a 
manner  that  is  not  observable  by  the  BTS  staff. 
For  example,  JFACC  team  members  interact 
with  each  other  using  handwritten  and  electronic 
notes  and  orders,  telephonic  communications, 
and  face-to-face  discussions.  It  is  simply  not 
possible  for  the  BTS  .  staff  to  monitor  all 
interactions  between  JFACC  members. 

The  prime  responsibility  for  gathering  JFACC 
process  and  performance  data  lies  with  the 
Analysis  Division  in  the  BTS.  They  work  with  the 
JFACC  staff  before  each  exercise  to  ensure  their 
training  requirements  are  clearly  identified  so  the 
BLUE  FLAG  training  scenarios  can  be  structured 
to  allow  the  JFACC  staff  to  be  stressed  in  key 
areas.  This  pre-exercise  analysis  gives  the  BTS 
staff  key  indicators  to  look  for  as  they  observe 
BLUE  FLAG  exercises.  Thus,  the  feedback  they 
develop  from  their  observations  tends  to  be 
directly  relevant  to  exercise  objectives.  In 
addition,  the  BTS  staff  attempts  to  observe  NAF 
behavior  that  is  not  anticipated  in  case 
behavioral  feedback  can  be  developed  based  on 
those  activities.  However,  developing  feedback 
based  on  either  anticipated  or  unanticipated 
activities  is  a  challenge  due  to  the  very  large 
number  of  activities  occurring  and  the  relatively 
few  observers  available. 

The  value  of  the  feedback  varies  depending 
upon  the  willingness  of  the  JFACC  staffs  to  use 
it,  and  their  ability  to  make  use  of  the  BTS  staff’s 
observations.  Some  JFACC  staffs  make  only 
limited  use  of  the  feedback  because  they  feel  it 
is  not  directly  relevant  to  what  they  perceive  their 
weaknesses  to  be.  In  some  cases,  the  JFACC 


staffs  may  see  the  relevance  of  the  feedback, 
but  may  not  be  able  to  make  immediate  or  full 
use  of  it  because  of  the  time  constraints  inherent 
in  an  on-going  exercise.  In  these  cases,  the 
post-exercise  feedback  is  very  valuable  because 
the  JFACC  staffs  then  can  reflect  on  the 
feedback  and  build  remediation  strategies  at  a 
more  leisurely  pace. 

Timeliness  -  A  long  established  principle 
of  effective  learning  requires  feedback  to  be 
delivered  to  the  learner  as  soon  as  possible  in 
most  cases.  The  nature  of  BLUE  FLAG 
exercises  makes  it  very  difficult  to  provide  the 
necessary  feedback  to  many  of  the  NAF  staff 
members  in  a  timely  fashion.  Normally,  there  is  a 
formal  feedback  session  provided  to  key  NAF 
staff  members  by  the  BTS  analysis  division  at 
the  end  of  each  day  and  to  Combat  Plans  on  a 
daily  basis.  In  addition,  BTS  staff  members  will 
also  give  feedback  to  BLUE  FLAG  participants  at 
certain  times  during  the  exercise  if  they  see 
something  that  is  a  serious  problem.  Generally, 
however,  the  BTS  staff  will  wait  to  provide 
feedback  until  a  logical  stopping  point  in  the 
exercise. 

The  BTS  staff  must  make  a  series  of  trade-off 
decisions  about  when  to  provide  feedback. 
While  it  is  advantageous  to  provide  feedback  as 
soon  as  an  error  occurs,  it  would  be  disruptive  to 
the  flow  of  the  exercise  to  continually  interrupt  it. 
After  all,  the  NAF  staffs  come  to  BLUE  FLAG  to 
get  a  realistic  command  and  control  experience. 
Providing  feedback  in  a  disruptive  fashion  would 
perhaps  bring  too  much  artificiality  to  the  training 
setting.  This  dilemma  is  not  unique  to  the  BLUE 
FLAG  exercises,  but  it  is  exacerbated  by  the 
number  of  participants. 

Manual  Nature  of  Current  Feedback 
Approaches  -  Currently  there  are  no  automated 
tools  to  help  BTS  analysts  in  developing  their 
feedback  comments.  The  analysts  use  paper 
checklists  and  notes  to  keep  track  of  their 
observations  and  to  formulate  feedback 
comments.  These  observations  and  comments 
are  kept  in  a  paper  archive  for  future  reference 
as  the  BTS  observers  prepare  final  reports  for 
each  BLUE  FLAG  exercise.  PowerPoint 
presentations  are  used  to  give  feedback  at  the 
end  of  each  exercise  day. 
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Large  Volume  of  Data  Generated  by  a 
BLUE  FLAG  Exercise  -  As  indicated  above,  one 
of  the  difficulties  in  generating  quality  feedback  is 
the  sheer  volume  of  data  produced  by  a  BLUE 
FLAG  exercise.  One  can  imagine  the  number  of 
digital,  voice,  and  written  communications 
passed  between  the  1200  plus  personnel  at 
each  exercise  over  the  course  of  a  week. 
Determining  which  pieces  of  data  are  important 
enough  to  examine,  collect,  and  catalog  is 
crucial  to  developing  useful  feedback.  The  BTS 
staff  has  extensive  experience  at  knowing  what 
to  look  for,  but  again,  the  large  amount  of  data 
means  there  is  always  a  chance  that  some 
important  data  may  not  observed. 

Potential  Feedback  Techniques  and 
Technologies 

Data  Capture  Tools  that  Combine 
Disparate  Streams  of  Data  -  Before  quality 
feedback  can  be  developed  in  any  learning 
situation,  it  is  first  necessary  to  define  which  data 
are  indicative  of  quality  or  poor  performance. 
The  BTS  staff  has  made  some  progress  in 
defining  which  criteria  are  critical  as  indicators  of 
a  quality  Air  Operations  Center;  however,  due  to 
the  complexity  of  AOC  activities,  it  is  very  difficult 
to  define  a  complete  set  of  criteria  that  are 
always  relevant  to  each  BLUE  FLAG  exercise. 
Recently,  the  Air  Force  decided  to  treat  an  Air 
Operations  Center  like  a  weapons  system  which 
will  lead  to  more  clearly  defined  performance 
criteria. 

As  described  above,  there  is  a  large  variety  of 
data  that  is  produced  by  the  many  teams  and 
sub-teams  which  participate  in  a  BLUE  FLAG 
exercise.  The  quantity  and  quality  of  feedback 
that  can  be  derived  from  an  exercise  is  directly 
affected  by  the  data  the  BTS  analysts  can 
capture.  Current  techniques  for  capturing  all  of 
the  relevant  data  are  limited  to  what  the  BTS 
analysts  can  see  and  hear.  What  is  needed  is  an 
automated  tool  or  tools  that  will  allow  the 
analysts  to  combine  what  they  see  and  hear  with 
digital  data  taken  from  the  exercise  when 
possible.  A  few  examples  of  the  type  of  data  that 
might  be  captured  and  then  used  in  the 
derivation  of  feedback  might  help  to  illustrate  this 
point: 


■  Data  plots  of  the  number  of  intelligence 
inputs  to  the  Air  Tasking  Order  planning 
process. 

■  Data  plots  of  inquiries,  along  with  their 
type,  from  one  sub-team  to  another 

■  Types  of  sorties  generated,  along  with 
their  success  rates 

There  are  many  other  examples  of  data  that 
when  analyzed  together  could  help  us 
understand  the  fog  of  war.  While  all  of  these 
types  of  data  can  probably  be  collected  right  now 
by  the  BTS  staff,  there  is  no  way  to  synthesize 
the  data  so  the  entire  AOC  “picture”  can  be 
seen.  By  seeing  the  picture,  we  are  not  talking 
about  looking  at  a  battle  map  of  the  theater,  but 
rather  examining  the  processes  together  which 
indicate  how  the  ATO  is  developed  and  how  that 
process  can  be  improved.  The  BTS  staff  needs  a 
capability  to  quickly  detect  and  analyze  data 
trends,  patterns,  and  anomalies. 

A  key  advantage  of  these  types  of  data  tools 
would  be  the  rapidity  with  which  the  feedback 
could  be  applied  to  the  JFACC  trainees.  It  may 
not  always  be  appropriate  or  effective  to  provide 
immediate  or  nearly  immediate  feedback  to  the 
NAF  personnel  because  BLUE  FLAG  exercises 
are  designed  to  be  realistic,  and  providing 
feedback  in  the  middle  of  the  exercise  would 
often  harm  that  realistic  nature.  However,  there 
are  times  currently  when  Battlestaff  Training 
School  staff  will  provide  feedback  on  a  more 
frequent  basis  than  merely  at  the  end  of  the 
exercise  day  if  they  see  the  JFACC  battlestaff 
make  errors  so  severe  the  end  objectives  of  the 
exercise  cannot  be  achieved.  With  the  data 
capture,  analysis,  and  feedback  tools  described 
in  the  section  such  immediate  feedback  would 
be  more  precise  and  timely  than  current  BTS 
staff  observations  allow. 

These  types  of  tools  would  also  greatly  aid  the 
quality  and  quantity  of  the  feedback  provided  to 
the  NAF  battlestaff  at  the  end  of  an  exercise  day. 
Whereas,  the  feedback  now  tends  to  be  in  the 
nature  of  verbal  briefings,  aided  by  manually 
constructed  PowerPoint  slides,  improved  feed¬ 
back  could  be  given  via  computer  graphics 
presentations  generated  from  data  automatically 
collected  as  the  exercise  proceeds.  Using 
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preplanned  templates,  the  BTS  analysis  division 
could  build  the  necessary  briefing  graphics 
quickly. 

The  most  important  benefit  would  be  the 
capability  to  allow  both  analysts  and  the  JFACC 
to  “experience”  the  data  from  a  variety  of 
viewpoints  and  perspectives.  Rather  than  merely 
show  “bird’s-eye”  views  of  the  theater,  with 
battle  units  identified  via  icons,  these  new  tools 
would,  at  the  touch  of  a  button,  show  trends, 
patterns,  and  anomalies  that  would  not  likely  be 
seen  by  the  BTS  analysts  by  merely  observing 
exercises. 

Finally,  these  data  capture  and  feedback  tools 
could  be  used  to  create  BLUE  FLAG  archives 
which  could  be  used  for  analysis  purposes  to 
detect  good  and  poor  Air  Operations  Center 
practices  across  Numbered  Air  Forces. 
Currently,  BLUE  FLAG  exercise  scenarios  are 
different  enough  between  NAFs,  and  the  data 
generated  by  each  exercise  is  so  vast  that  it  is 
not  possible  to  synthesize  exercise  processes 
and  products  on  a  large  scale.  Therefore, 
analysts  cannot  adequately  compare  and 
contrast  AOC  approaches  between  the  NAF 
battlestaffs  that  participate  in  BLUE  FLAGs. 


Conclusion 

The  Air  Force’s  Battlestaff  Training  School  has 
evolved  a  highly  sophisticated  and  successful 
approach  to  training  large  battlestaffs.  An 
indication  of  the  BTS’s  effectiveness  is  shown  by 
the  fact  that  Joint  Force  Air  Component 
Commanders  continue  to  request  opportunities 
to  work  with  the  BTS  in  BLUE  FLAG  exercises. 
As  has  been  proven  many  times  in  past  wars,  a 
military  force  is  no  better  than  its  command  and 
control  capabilities,  despite  its  size  or 
technological  prowess. 

Despite  its  current  effectiveness,  the  BTS 
Commandant  and  staff  are  always  exploring 
new  methods  and  technologies  to  keep  the 
BLUE  FLAG  exercises  on  a  path  of  continuous 
improvement.  Examinations  of  how  best  and 
when  to  link  live  and  virtual  assets  into 
wargaming  exercises  will  help  to  strengthen  the 
degree  of  realism  BLUE  FLAG  trainees 
experience.  In  addition,  better  ways  to  capture 
and  provide  quality  feedback  to  battlestaffs  will 
prepare  them  more  completely  for  challenges 
they  are  likely  to  face  in  war. 


